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ABSTRACT 

This report summarizes the discussions at a seminar 
which provided the opportunity for 15 researchers and developers from 
the United Kingdom and other European countries to consider a number 
of short, medium, and long-term issues and assist in setting an 
agenda for future phases of research. The specific goals were: (1) to 
identify the tools necessary for the effective support of existing 
authors or development teams of computer-supported learning or 
training materials (short term goal); (2) to indicate where advanced 
developments in this and related fields might lead to better 
computer-based training development tools (medium term goal); and (3) 
to suggest areas of fundamental research which are needed to underpin 
more effective courseware development tools for the future (long term 
goal). General issues covered included computer-based training and 
users of authoring tools. Several topics related to what tools are 
needed to improve current practice are then considered, i.e., the 
limitations of current authoring systems; problems to be solved with 
better tools; assessment of organizational needs; analysis of 
training needs; design: prototyping tools; and user modelling. 
Software engineering techniques are described, including simulation 
and modelling, Smalltalk and direct-manipulation interfaces, 
HyperCard, integration, and expert systems. Several issues of 
instructional and learning strategies are summarized. The report 
concludes with outlines of looIs and techniques that need to be 
developed and issues for the research on which such development 
depends. A list of seminar participants is appended. (MES) 



********************************************************************* 



* Reproductions supplied by EDRS are the best that can be .aade 

* from the original document. 



EISIRIC 



h "XMOMKANO SOOAL MSEAfKH COUNCIL 

K U S OFPAHTMCKT OF EDUCATION 

|- /"J^ Cttce ol Cduc«tK>o«i Research and Impfovement 

^ EDUCATIONAL RESOURCES {NFORMATtON 

I CENTER (ERtQ 

I' This documenj has b«en reproduced as 

^ received 'rom the pe'SOn or OfQanization 

I CO Of<9»natir>Q it 

^ □ Minor changes have been made to improve i| 

r (■■■^ <^eproduclion quadty ^ 

I •« a Points Of v«ew or optnioos slated m this docu- 

^ '"•"t do r>ot r>eces»«riJy represent ofticisi 

^ OERI position or policy 



Support Tools for Authoring 
a seminar report 



Occasional Paper InTER/7/88 
December 1988 



u 



\^ "PERMISSION TO REPRODUCE THIS 

i MATERIAL HAS BEEN GRANTED BY 

t R. Lewis 

\ Edited by: 

I PROFESSOR R. TEWIS 

I UNIVERSITY OF LANCASTER 

^ AND EDUCATIONAL RESOURCES 



rpTX XMArrr^ INFORMATION CENTER (F JC) 

INFORMATION LB SYSTEMS LTD 



II nIERI 



IhfFORMATION TECHNOLOGY* 
IN EDUCATION RESEARCH? 
I i' ' ^ I ft — I \| PROGRAMME^ 
i ERXC''^^ PSYCHOLOGY, UNIVERSITY OF LANCASTER, LAI 4YF^ 



Origins of the ESRC 
INFORMAnON TECHNOLOGY AND EDUCATION 
PROGRAMME 



The Education and Human Development Committee was established with the 
reorganisation of the Uien Social Science Research Council in May 1982. In 1984 
the Council changed its name to the Economic and Social Research Council. 
Early in 1983 the Committee identified and circulated for discussion an initial 
listing of important topics which waiianted expanded support or accelerated 
development. The broad area of Infcrmation Technology in Education occupied a 
prominent place in that list. The Committee emphasised its intention that research 
would be centred not only on thf effect on education of machines to help teach 
the existing curriculum, but on the development and adaptation of the curriculum 
to equip people, including those of school age, to deal with intelligent machines 
and to prepare them for a lifA changed by their arrival. For example, there are 
questions concerning both cognitive and organisational factors which facilitate or 
inhibit the adoption of Infcrmation Technology in Education, and allied to these, 
questions around the navure, characteristics and development of information 
technology literacy. Th^se initial topics remain central to the Committee's 
projected agenda. 

Two reports were com'nissioned and detail'id discussion and workshops were held 
in 1983. In its further considerations, the Committee was conscious of the fact 
that the research community is widely scattered and has relatively few large groups 
of researchers. Furthermore, it recognised the importance of involving practitioners 
and policy makers in the development of its programme of substantive research 
and research related activities and the necessity of ensuring close collaboration with 
commercial organisations such as publishers, software houses and hardware 
manufacturers. It was this thinking that led the Conunittee away from the 
establishment of a single new centre to the appointment of a coordinator as the 
foca' point for the development of the initiative throughout the country. 

The brief for the Coordinator included: 

- the review, evaluation and dissemination of the recent and current activity 
in the Held of Information Technology and Education; 

- the identification of the needs of education in relation to Information 
Technology; 

- the stimulation of relevant research and the formulation of research 
guidelines; 

- the establishment and maintenance of a database of relevant work and 
undertaking arrangements for coordinating and networking of those active in 
the field including cognitive scientists, educational researchers, practitioners 
and policymakers. 

In January 1988 the Council of ESRC approved a new initiative which would have 
resources to support a substantive research progranmie. This programme, the 
Information Technology in Education Research Programme, started in the autumn 
of 1988. The new series of InTER Programme Occasional Papers has a similar 
format to the previous ITE Programme series and covers aspects of the 
Programme's work. These are listed on the back cover of this paper. 
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1. INTRODUCTION 

This Occasional Paper* is the report of a seminar organised by the ESRC- 
Information Technology in Education Research Programme. The overall objective 
for this and other similar seminars is to assist the Programme in setting an agenda 
for future phases of research. 

About 15 researchers and developers from the UK and other European countries 
(see Appendix A) participated in a 24-hour seminar in September 1988. The 
ESRC-ITE/Training Commission study report on Authoring of Computer-based 
Training Materials (rrE/27/88) was distributed to participants as a source 
document. At the seminar there were five sessions: three consisted of open-forum 
discussion; an evening session examined a range of 'advanced' softwa^-e 
environments from the UK, France and Norway and North America; the fourth was 
conducted partly in smaller groups with reports back to a final plenary session. 

The specific goals of the seminar were to review a number of short, medium and 
long-term issues: 

- to identify the tools necessary for the effective support of existing authors 
or development teams of computer -supported learning or training 
materials, (short term); 

- to indicate where advanced developments in this and other related fields 
might lead to better CBT^ development tools, (medium term); 

to suggest areas of fundamental research which were needed to underpin 
more effective courseware development tools for the future, (longer term). 

This report does not attempt to give an account of the seminar in a time -ordered 
sequence but structures the points raised in relation to the goals stated above. 
Understandably, it has not been possible to come to a consensus of views on every 
suggestion or concern. In many instances, the points were raised by individuals or 
a minority of participants. Where dissenting opinions were voiced, however, they 
have been reported. 

GENERAL ISSUES 

There was an early discussion on the general issue of the place of CBT in general 
education as well as specific comparisons of CBT with other forms of vocational 
training. The discussion then focussed on the users of authoring systems. 

11 Computer- based training 

CBT shares with other forms of education the necessity of confronting some fairly 
intractable problems. Mixed-ability teaching, for example, hard for human 
trainers and educators to do, let alone a computer system."" Training in general also 
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has to cope with the existing structure of an organisation, and to acknowledge that 
the provision of training will often change that organisation. 

Another issue - one that surfaced under various guises - was the problem of 
'making exphcit' the objectives of a training course, whether that training is 
computer-based or otherwise. For example, ''characterising the learner requirements 
is a general educational problem, but it is especially relevant in CBT, where there is 
usually no interaction (between designer and learner) at the time the material is 
delivered" and ""frame-based CBT embodies the objectives of the author and training 
manager, but these are not made explicit. The authoring system does not ask, ' What 
are your objectives for this piece of training material*, no" does it have the ability to 
categorise or use this information" "People don) learn effectively if there is no 
structure; learning has to be based on previous knowledge. People need to be able to 
make mistakes, and receive support; there is very little learning without that 
interaction. This needs organisation, structure and strategy', which can only come from 
ihe teaching objectives" 

CBT, however, has special abilities and special problems compared with other 
forms of training: 'CBT is re-usable", ''CBT can reduce by half the time it takes to 
become proficient" and "CBT can encapsulate knowledge which may be very scarce 
within an organisation, and make it more widely available." Whilst other training 
mechanisms may demonstrate some of these benefits, the rigour demanded by CBT 
programmes requires a more logical and detailed analysis of the interaction with 
the trainee. 

Although it was generally agreed that CBT was a long way from fulfilling the 
apparently modest objective of providing "at least as good as the worst human- 
student interaction" y it has to be judged against its own objectives, "h enables new 
hnJs of interactions - don't :.y to compare it with human tuition" 

The discussion of CBTs special problems led to a consideration of how the 
process of producing courseware could be improved. 

2.2 Who wdl be the users of authoring tools? 

A major aim of the fii-st session was to identify the target users for authoring 
systems. "Weve tried to make authoring systems available to teachers. But authoring 
should be done by professionals. If you design software for use by novice authors, it 
gets in the way of experienced software developers. We need different toolkits for 
different levels of user" 

Some felt that toolkits should be provided for subject-matter experts: ''teachers 
know what they want to teach\ but that they would prolably need a fairly 
supportive environment. "For teachers, allow them to structure their ideas, especially 
with a courseware map" and, "novice authors need help with tactics and strategy." 
More experienced authors, however, "want to be able to be free of these constraints; 
they need a freer- format system" 

One solution discussed was to provide a 'non-prescriptive' system, one where "all 
methods are available at once, and the tutor can decide which ones to ignore." This 
feature is reported to be present in some systems currently under development. 
However, "we need to consider the trade-offs between productivity and flexibility. The 
more highly-specialised a tool is, the more productive it will be within its area of 
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speaalisaiion. Flexible systems (which can address many different kinds of problem), 
may not be excellent at solving any of them" 

Other than subject-matter experts and professional courseware developers, the users 
of si'ch tools may even include the trainees themselves, or at least their managers: 
"many clients (for example, lie Na\y) av/c for the ability to make changes themselves". 

On the other hand, perliaps Ve should concentrate on building tools which enable 
us to build the environments, and the tools to use those environments!' There was 
no consensus on the approach to take and opinions varieo from the view that 
easily accessible systems were needed, to the standpoint that, ""simple solutions for 
complex problems are 'isually inadequate." 

Participants had different preoccupations. In particular, some advocated the 
provision of authoring toolkits for children: 'V/ie fact that learners have access to 
loob whtch enable them to build models means that they have to son out the 
knowledge representation, and understand the urderlying theory"" and, 'it's no good if 
only experts can produce (software) models. Also, "there are not enough toots to 
allow learners to build software models.^ 

To sum up. ^ Users of authoring tools will vary enormously. Usually, a team of 
peoplp will be involved in authoring: each person may well need different tools" and. 
We *ieed a senes of different torls A tool for building simulations may not help 
with problems in knowledge eliciiation." 

3. WHAT TOOLS ARE NEEDED TO IMPROVE CURRENT PRACHCE? 

Support tools may be considered at three levels: 

- those which speed up the implementation or coding process and hence 
boost the productivity of experienced developers; 

- those which aid 'low level' preparation of text and graphics (spelling and 
style checkers, WYSIWYG (what-you-see-is-what-you-get) interfaces, 
paintbox/ drawing tools, and so cn), which can greatly assist the non- 
expert: 

- those which support high level educational/training design of CBT 
materials, and of sophisticated user interfaces. This level of expertise is 
currently oiilv found in the most well trained and experienced human 
minds. 

3.1 The limitations of current authoring systems. 

Manv deficiencies were discussed. For example, "current authoring languages dont 
allow mt to do much more than a conventional programming language would They 
don't provide a 'theoretical basis' for CBT; they merely provide an environment for 
defining .screen displays and mtcrauions." Other common complaints were a lack of 
support for the design stage of the process and the sheer lack of productivity of 
the existing systems 

"Tfierc are no tools which help the design phase of the process." ''Although the 
presentation is getting better, there is no help in designing the experience the user will 
go through" and, 'Taper -bused design is the state-of-the-art; it must go We need 
'^ols that allow us to express the dt^ign, and to code it, without paper." 
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"Anything that makes tht process of turning an idea into screens of interaction, and 
then into code, will be useful Mostly, we try to break out of the constraints that the 
existing systems impose upon us." 

3.2 What problems can be solved with better tools? 

The distinction between what a tool can and cannot do provoked much discussion. 
"Tools only allow you to do existing things more effectively,'' "Tools don't help with 
the user expaience • it's human intellect that does that!* More generally: Tom can't 
expect tools, however clever, to substitute for the intellectual process. Certainty, we 
would prefer to sperui 150 hours to produce courseware rather than the 300 we spend 
at the moment. But each of us have more or less skill and ability to produce 
effective material and the quality of the product will always depend on that" 

Some participants, however, felt that there were counter-examples: '14 tool like 
NoteCards actually albws you to structure your ideas and arguments, it's not just a 
word processor. It augments your abilities."* 

Some of the specific areas where tools were felt to be required are discussed in 
the remainder of this section. 

3.3 Assessing the needs of the organisation. 

As has been mentioned earlier, any kind of training must take into account the 
pre-existing structure of the organisation, /ui important part of designing a 
training course, therefore, can be to investigate the structure and dynamics of the 
existing organisation. Some techniques exist: for example, to try to identify the 
trainer or part of the course whirh causes difficulties: "The 'odd one out* 
technique. Look for the eccentric bit - that may well be why the training problem 
exists in the first place" and, you can analyse areas of conflict, this helps to identify 
the common ground."" So far, however, there are no tools to help with this process. 

3.4 Training Needs Analysis 

One participant reported problems m assessmg the level of ability of the target 
trainees. ''The problem is that all our courseware contains a large proportion cf 
redundant material, because we cannot identify the true abilities of the trainees as they 
enter the course. Often, we find that the training managers that we talk to 
underestimcue the lex'el of ability, literacy, competence, and so on, of the target 
population; we find that they're much bnghter and more able than we had been led 
to believe!* 

"A simple pre -test is not sufficient, because it doesn't take into account the fact that 
an individual may be at different leveb of ability or experience in the various areas of 
the subj'^ct. We need a tool to help us do the analysis of the target population, to 
identify thc'r true stoning points, and to minimise the redundancy!' This appeared to 
be a common problem: "IBM insist on talking to the trainees, interviewing them to 
find out their true ability." One participant saw an historical parallel: "Thirty years 
ago, systems analysis was stressed as crucial, now Educational Systems Analysis, even 
the basic Training Needs Analysis is not emphasised enough." 

3.5 Design 

The need to elicit, structure, and represent the body of knowledge involved in the 
course, and the ways in which it might be communicated to the student, currently 
^ suffers from a lack of support. In particular, "most existing authoring tools are 
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languages*', offering no real guidance on how to structure a course or a series of 
interactions. Where guidelines do exist, they are not part of the tools. Authors of 
CBT should work at a high level of abstraction when they embarJr upon CBT 
construction; too often their preoccupation is with the nuts and bolts of the 
training rather than taking a more distant, overall view from the start. But how 
should this overview be attained? 

The experience of the Open University was cited. Teams work in a well defined 
area when they produce OU course texts. The texts have a distinctive ^tyle 
consisting of objectives, aims, in- text questions, self- assessment questions, etc. and 
there is a recognised 'correct' way of putting the texts together, starting with the 
objeaives. In practice, of course, not all authors follow recognised good practice 
in achieving the end product. Guidelines may exist but the underlying theoretical 
aspects of design are achieved through secondary actions rather than as a 
fundamental framework from the start. Academics are concerned to teach their 
subject matter and to explain it to the best of their ability; they do not start ty 
working at a higher level of abstraction about how the various parts of the subject 
matter should link tof;ether. They may end up there, but they don't start there. 
If this doesn't happen when texts are prepared, can we expect it to happen during 
courseware production? One might even ask if we wish to promote such methods. 
Flexibility in authoring tools, provkling a supportive environment in which ideas 
may be expressed and refmed as production proceeds may be an appropriate way 
ahead; in other words, tools for flexible design and rapid prototyping are required. 

The point was re -iterated thai tiie existing software tools do not incorporate a 
methodology for courseware production, except by default: "Te/icone has a set of 
interactions that it supports, but we need a wider range of interactive style:" 

Once the course layout has been designed, the presentation of the screc^ 
themselves is also dependent on the individual skill of the authors. "W? need 
guidelines to put screens together^ - although, once agam, whether a tool could help 
achieve this was disputed: 'If you want help with presentation, go to an art college, 
not to the art matenab supplier!" 

However, most people felt that such tools would prove useful: "Advice on which 
colours to use, or a fog index (an assessment of the literacy level required to read 
a piece of text, based on the average number of syllables in the words)." Such 
systems, however, should provide advice, rather than simply be a mechanical 
process: 7n all the books on 'How To Produce Good CBT, all you get is examples 
of Bad CBT. Tell us what Good CBT b, and incorporate it'\ and ""something like 
'Style' on UNIX doesn't give you any advice, it simply points out what's wrong" 

3.6 Prototyping tools. 

The low productivity of the existing tools has several important side- effects. 
Firstly, many training needs are simply not being addressed at all: Remember that 
the cost of a solution must be in relation to the cost of the need' and, "we need to 
train probiem -solving skills'" but, "simulations are beyond the scope of existing 
authoring systems, you haxe to incorporate calls to a general -purpose programming 
language*' 

It also means that when courses are finally constructed, it is difficult to justify 
changing them. "What usually happens is that the courseware is delivered late and 
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there are great incentives not to make any last -minute changes because they are 
diffcuh to integrate" 

Most delegates wanted the situation to improve: "We shouldn't necessarily o^ct that 
courseware remains in the curriculum for long periods - in some cases we need better 
tools to produce 'throwaway* courseware, so that it can be re-written frequently!' 
"H^e need a fast turnaround time on the interface, so that we can make changes 
more easily, so that the problems of adaption are not at that level." 

As in modem software engineering, the benefits of a 'rapid prototyping' 
environment are not confmed to the speed with which applications can be 
generated; the ability to produce iterative designs allows the developer and user to 
communicate more effectively. But modularity is also important: "We need 
prototyping toob. We need a system which enables us to make changes late on in 
the development process." 

3.7 User modelling. 

'Adaptive CBT' is not a new idea; prototype systems, and indeed production 
systems, incorporating sophisticated branching strategies have been produced for 
some time. One application which had been constructed "had 100 concepts, and 
* stereotype* user groups as starting points As you went through the course, you 
modified the user model if the response didn't fit." 

But it is still not generally cost-effective to incorporate such modelling in 
commercially- available CBT. To put it another way 7^5 feasible to a certain level 
of granularity - // you can identify perhaps four or five paths through the course, it's 



Some participants, however, questioned the advisability of pursuing this goal too 
hard. V« cases where the solution turns out to be more and more compkx, often 
the best thing to do is to decide that the solution lies * somewhere else' - look for 
alternative methods of achieving your original goal." This brought the discussion 
back to the problem of making the knowledge explicit: "CBT diagnosis is at the 
* content*, rather than the cognitive level." 

This topic, of course, is the subject of much lively debate elsewheie 



4. WHAT SOFTWARE ENGINEERING TECHNIQUES ARE USEFUL? 
4.J Simulation and modelling 

The topic of modelling of processes as part of courseware was discussed many 
times. ''Better courses, or individual lessons, often come about because people say, 
'there is a different way of teaching this topic \ We need more models, for example 
simulation; most CBT hasn't used simulation." 

The reasons for the existence of little simulation- based courseware were also clear. 
"The production of significant simulations is still hard and its success depends on the 
skill of the developer in modelling the ideas of the domain''', and "there is no clear 
methodology for how to develop and use models - w^ need to provide help and 
guidance for people trying to do it." 
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It is, however, worth doing: "As you move from simple simulations to more complex 
ortfis, tF^ computer becomes much less of a * presentation* device and mcreasingly 
has more 'awareness* of the subject-matter This can provide a richer level of 
interaction, based on the model of the domain." 

4.2 Smalltalk and dirert- manipulation interfaces. 

Man> participants had experience with Smalltalk, both directly and by being 
influenced by its underlying approach. One idea in particular which provoked 
discussion is the ' Model - View - Controller ' paradigm. 

As applied to CBT courseware, the following (outline) definitions emerged; 

The 'model* is the knowledge or behaviour that is codified. It may be a 
mathematical model, or data, or heuristic or logical rules; it be may 
considered as the 'content' of the courseware. 

The 'view' is how that knowledge is presented to the user - the 
'presentation* of the course. 

The 'controller' is the mechanism by which the user affects the sesaion - 
the 'interaction'. 

This approach encourages modularity - the content, the presentation, and the 
interaction with the student should be considered as three separate design issues, 
any one of which can be changed without affecting the other two. 

This type of software environment offered new possibilities. "The Alternative Reality 
Kit (a software environment which allows users to gain an appreciation of physical 
laws such as gravity, by presenting them within a 'world* where the rules can be 
changed), offers something that couldn't be done using traditional methods." 

4.3 HyperCard. 

This product, available on Macintosh compu:ers, has begun to be used for 
authoring, with some success. HyperCard removes a limitation; it gives us an area 
where we can improve our service. We found that we could produce CBT frames in 
half to two- thirds the time it normally takes us." 

There are limitations. "On the Macintosh the screen is black-and-white, and the 
system doesn't provide artswer matchmg facilities", but there are compensations: ''We 
can zoom into the material and allow the trainee to jump around a body of 
knowledge with greater freedom, and so on. It allows us to give solutions to training 
needs thai were too costly before.** 

Another big advantage conies when the client changes his mind at the last 
mon^ent. 'With HyperCard, the material is divided into identifiable chunks of 
information, so that we can make changes to one part of it without altering the 
underlying structure." 

4.4 Integratioru 

This theme appeared in many contexts. "We should resbt the temptation to make 
tools do more than the jobs they were originally intended for. TItat just leads to bad 
tools. What we should aim for is better integration between tools, so that a number 
of special-purpose tools can exist and work well together."* 
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4.5 Expert Systems. 

One participant reported that a system incorporating expert system techniques was 
beginning development. "77ie system aims to cope with both the initial, and the 
continuing training needs. It u'orfej in two 'modes' - an advisory mode, which 
consists of an expert system which can give advice on what to do under particidar 
conditions, and a draining' mode which can teach the concepts required in order to 
make the decisions. We expect that incidental learning will take place while the 
system is in the advisory mode - this is quite often a time of great stress." 

As usual, however, there are dangers: ''There are at least two problems with using 
expert systems for trainmg. Firstly, when you introduce them, you can disrupt the 
normal process by which people get better at problem-solving - that is, through social 
mteraction with peers {eg. anecdotes) which provides updates to a changing knowledge 
b'lse ' a knowledge ba,e which is usually distributed amongst a community of experts 
rather than residing in one or two individuals. Secondly, you have the problem of 
actually making sure that trainees are really doing what you tell them to. If the 
expert system scys. 'Look at component A. Is it faulty?*, and the trainee answers 
*Yes\ \vu don V know whether she really has looked at the correct component.*" 



5. ISSUES OF INSTRUCTIONAL AND LEARNING STRATEGY. 

Almost all the discussion introduced earlier (which tools were needed and should 

be produced), pointed to some fundamental research which was needed in order to 

evaluate wh;* has been done, and to formulate categorisations and guidelines based 

on what constitutes 'good practice* in the various areas. These are listed in 

Section 6. Also, research is needed which may lead to a taxonomy of instructional 

strategies. 

The assertion that current courseware had an impoverished set of styles m which 
ihe trainee could interact led to the notion that we might provide, 'facilities for the 
author to help decide what sort of user interface to create - what style of interaction!" 
Very often, "these types of intenu.tion are embedded in courseware as a * house 
style'", but what we need is \i language to express the different kinds of interaction". 
to be able to sav. "thus class of problem %ccms to be best tackled by this approach." 

Once a taxonomy of student interactions has been established, "we should provide a 
high-level interface for designing sessions bused on them. For examplcy if we decide 
that a Socratic dialogue « an appropriate method^ the system should provide a 
framework for that." Others argued that the decision could also be left to the 
user. "Let the trainee choose, say, frame -based versus simulation learning styles." and 
account should be taken of, for example, holistic and serialist learners. 

In any case, one parucipant thought that "what we are nussing is considering the 
thing from the student 's point of view. Current authonng systems consider things from 
the author's point of view - we should start with the student view, with the idea of a 
' learning transaction ' - and then design the tools from there"' 

Instructional strategies at the 'macro' level should also be investigated, and 
guidelines produced, for example, "ishcn to intervene, s^hcn to allow the student to 
carry on making mistakes, etc!' 

12 
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6. TOPICS FOP. RESEARCH AND DEVELOPMENT. 

As indicated in ihe Introduction to this paper, the main objective of the seminar 
was to assist the InTER Programme in defining an agenda for research in the area 
of support tools for the authoring of CBT courseware. Perhaps more than in 
other areas of the Programme's concerns, the distinction between research and 
development in not distinct in this field. It may be argued that toois to support 
training needs analysis would be valuable in training materials definition and design. 
What is not at all clear is whether we have sufficient understanding of trairung 
needs analysis so that the development of the necessary tools can be undertaken 
without further research. 

Despite this difficulty, participants worked in small groups during the fine? session 
of the seminar and attempted to identify both key issues for basic research and 
tasks which required development effort. The distinction was not taken as a purely 
academic exercise, but rather to identify where the various agencies concerned with 
authoring environments should focus their efforts. 

6.7 Toob and techniques which need to be developed: 
These may t)e divided into tools which: 

a) speed up or make the production process more efficient; 

b) should result in more effective learning through CBT. 



al io assist the stages from design to implementation; different tools for various 
parts of the process; 

a2 to help members of the development team to communicate more effectively, 
and to document the system (not necessarily paper- based); 

a3 to make it possible to take the output of one tool and use it in another 
environment for which it was not originally intended; 

a4 to feature the model -view -com roller paradigm which encourages modularity; 
the content, the presentation, and the interaction with the student should be 
considered as separate, any one of which can be changed without affecting the 
other two; 

a5 to produce ' metatools ' - tools for producing tools - based on a 
programming environment which is object-oriented, easily-extensible, etc.; 

a5 to build advice systems, for example, principles of graphic design for 
courseware authors, or advice on building models, so that individuals who are 
subject-matter experts can produce effective CBT. 

bl for training needs analysis; 

b2 to help highlight the form of interaction (the 'transactions') between the 
student and the system; 

b3 to help the knowledge elicitation process, (eg. structured conversation 



analysis). 
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d2 Issues for Research on which development depends. 

There is clearly a danger when considering research issues to identify general issues 
in the understanding of learning. This is not the goal of this paper. Rather it is 
to highlight somewhat more tractable questions which relate directly to the 
concerns of authoring environments and the specific opportunities offered by CBr. 

The issues are divided into: 

c) those which focus on the learner and a view of the learning process; 

d) those which relate to the development of CBT materials. 

cl understanding the way in which learners change their knowledge 

representation over time (how the structures change); 
c2 categorising the different metaphors for user interaction, in terms of 'this 

metaphor seems to be appropriate for this kind of interaction'; 
c3 identifying principles of good instructional strategies and formulating 

guidelines, eg. when to intervene, when to allow the student to cany on 

making mistakes, etc. 

c4 evaluating different learning styles against different training problems (both 

from the author*s and the trainees' viewpoints); 
c5 categorising the strengths and weaknesses of different instructional styles to 

arrive at statements such as *This class of problem seems to be best tackled 

by this approach ' ; 
c6 developing a taxonomy of instructional strategies; 

dl evaluating and codifying the underlying theories which will enable courseware 
'critiquing' tools to be developed for CBT design and presentation standards; 

62 evaluating methodologies and knowledge engineering tools; 

d3 work on treating authoring as the process of producing abstract structr.res - 
of representing the knowledge of the domain - and then using delivery 
systems which can embody knowledge in a behavioural sense, of how to 
formulate the interactions with the student; 

d4 developing metaphors for authoring environments which can be easily grasped 
by trainers (as opposed to programmers) and lead to the effective use of 
powerful tools by a wider, and often more sensitive, community; 

d4 analysing the existing tools, and categorising their mechanisms in terms of the 
model-view-controller paradigm. 
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